The present invention relates generally to computer networking, and particularly to direct memory access over a computer network.
As high-speed networks make it easier to use distributed resources, it becomes increasingly important to allow a device at one node of a computer network to directly access remote memories at other nodes of the computer network. By using Remote Direct Memory Access (xe2x80x9cRDMAxe2x80x9d) for data transfer, processors in a computer network may exchange data directly between their respective user memories with a minimum of intervention from the operating system. This leads to a significant reduction in the startup latency, and higher throughput.
An RDMA operation is a direct exchange of data between the memory contents of two devices over a network. RDMA means that once a node (e.g., a server) allocates, pins down (making it non swappable) a bulk of memory and sends the memory description to another node (e.g., a client) the other node can read and write to that memory directly. Directly means that the central processing unit (xe2x80x9cCPUxe2x80x9d) on the server or the client is not involved in the RDMA operation. With RDMA a client can manage the memory surrendered by a server as an alternative page file.
Accesses through a network always has possibilities of error. In order to make sure that a write access using RDMA is done properly, the device which performs the write access may have to perform a read access to read the written data. Also, in order to make sure that a read access using RDMA is done properly, the device which performs the read access may have to perform the read access again and compare the read data from the two read access. These methods for RDMA assurance are time consuming and waste network resources.
Therefore, there is a need for a system and a method that allows an initiator of an RDMA to obtain access assurance that the RDMA is done properly, and/or to obtain error information if an erroneous situation occurred during the RDMA process, without going through the aforementioned procedures.
The present invention comprises a method and system that provide access assurance regarding an RDMA transaction over a computer network.
In one embodiment of the present invention, a first device and a second device are coupled to first and second peripheral component interconnect (xe2x80x9cPCIxe2x80x9d) buses, respectively. The first and the second PCI buses are coupled to the computer network through first and second PCI network adaptors, respectively. The first and second PCI network adaptors include functional units to facilitate a memory-mapped read or write on the first PCI bus to be bridged to the second PCI bus through the computer network. An RDMA space and an associated access assurance space are assigned to the second device in the memory space of the first PCI bus, to which the first device is coupled. The first device may RDMA the second device by directly reading from or writing into the RDMA space assigned to the second device. The first and second PCI network adaptors process the RDMA transaction. For each RDMA transaction initiated by the first device and targeted at the second device, the second PCI network adaptor also prepares access assurance information corresponding to the RDMA transaction and sends it to the first PCI network adaptor in a response packet. The access assurance information is stored in the first PCI network adaptor until it is checked by the first device. To obtain this access assurance information regarding the RDMA transaction, the first device performs a PCI read from the assurance space associated with the RDMA space assigned to the second device in the PCI memory space of the first PCI bus.